Papers with neural network architectures
Structure Learning for Neural Module Networks (D19-64)
Copied to clipboard
| Challenge: | Neural Module Networks are a class of neural networks that involve human-specified neural modules . current models only learn the parameters of the modules and/or the order of their execution . |
| Approach: | They propose to learn internal structure and sequence without extra supervisory signals . they use dynamically composable modules which are then assembled into a layout . |
| Outcome: | The proposed model performs comparable to models using hand-designed modules. |
Pretrained Transformers for Text Ranking: BERT and Beyond (2021.naacl-tutorials)
Copied to clipboard
| Challenge: | This tutorial provides an overview of text ranking using neural network architectures known as transformers. |
| Approach: | This tutorial provides an overview of text ranking with neural network architectures known as transformers. |
| Outcome: | This tutorial provides an overview of text ranking with neural network architectures known as transformers. |
Start Simple: Progressive Difficulty Multitask Learning (2024.naacl-srw)
Copied to clipboard
| Challenge: | a novel neural network training strategy that trains neural networks using subtasks of progressive difficulty is proposed . this strategy could help us understand how neural networks learn, authors say . |
| Approach: | They propose a multitask learning strategy that employs progressive difficulty subtasks to train neural networks. |
| Outcome: | The proposed strategy can improve model performance across a range of NLP tasks and data sets. |
Character-Based Models for Adversarial Phone Extraction: Preventing Human Sex Trafficking (D19-55)
Copied to clipboard
Nathanael Chambers, Timothy Forman, Catherine Griswold, Kevin Lu, Yogaish Khastgir, Stephen Steckler
| Challenge: | Illicit activity on the Web often obscures information between client and seller, such as the seller’s phone number. |
| Approach: | They propose to use a dataset to model adversarial noise in a text extraction system and propose a visual character language model to interpret unseen unicode characters. |
| Outcome: | The proposed model improves number recognition by 89% over a CRF with a CNN and shows that unicode characters can be translated to unicoding. |
The Amazing World of Neural Language Generation (2020.emnlp-tutorials)
Copied to clipboard
| Challenge: | Recent years have seen a paradigm shift in neural text generation due to advances in deep contextual language modeling and transfer learning. |
| Approach: | They will discuss how and why NLG models succeed/fail at generating coherent text. |
| Outcome: | This paper will discuss how and why these models succeed/fail at generating coherent text, and provide insights on several applications. |
A Neural Pipeline Approach for the PharmaCoNER Shared Task using Contextual Exhaustive Models (D19-57)
Copied to clipboard
| Challenge: | NER and concept indexing perform named entity recognition and concept identifiers (CUIs) in a knowledge base. |
| Approach: | They propose a neural pipeline approach that performs named entity recognition (NER) and concept indexing (CI) they use bi-LSTM to capture the semantic information of a sequence and classify them into entities or no entities . |
| Outcome: | The proposed approach performs named entity recognition (NER) and concept indexing (CI) which links them to concept unique identifiers (CUIs) in a knowledge base. |
Exploiting Discourse-Level Segmentation for Extractive Summarization (D19-54)
Copied to clipboard
| Challenge: | Existing approaches to extract summarize text are based on sentences as the elementary unit, but semantic segments containing supplementary information or descriptive details are often nonessential in the generated summaries. |
| Approach: | They propose to exploit discourse-level segmentation as a finer-grained means to more precisely pinpoint the core content in a document. |
| Outcome: | The proposed method improves extractive summarization performance on CNN/Daily Mail dataset. |
Box Embeddings: An open-source library for representation learning using geometric structures (2021.emnlp-demo)
Copied to clipboard
Tejas Chheda, Purujit Goyal, Trang Tran, Dhruvesh Patel, Michael Boratko, Shib Sankar Dasgupta, Andrew McCallum
| Challenge: | Recent studies have explored alternative vector representations with different inductive biases or capabilities. |
| Approach: | They propose a Python library that extends probabilistic box embeddings to geometric shapes and regions. |
| Outcome: | The proposed library is fully open source and compatible with PyTorch and TensorFlow. |
Augmenting Neural Networks with First-order Logic (P19-1)
Copied to clipboard
| Challenge: | Existing paradigms for training neural networks require large datasets, a paper argues . we present a framework for introducing declarative knowledge to neural networks . |
| Approach: | They propose a framework for introducing declarative knowledge to neural networks . they compile logical statements into graphs that augment a network without extra learnable parameters or manual redesign. |
| Outcome: | The proposed framework improves on three tasks, especially in low-data regimes. |
Domain Expert Platform for Goal-Oriented Dialog Collection (2021.eacl-demos)
Copied to clipboard
| Challenge: | a prerequisite for the creation of a goal-oriented neural network dialogue system is a dataset that represents typical dialogue scenarios and includes various semantic annotations. |
| Approach: | They propose a web-based platform for collecting and writing goal-oriented dialogue samples. |
| Outcome: | The proposed platform is language-independent and is currently being used to collect dialogue samples in Latvian . |
Incremental processing of noisy user utterances in the spoken language understanding task (D19-55)
Copied to clipboard
| Challenge: | triggered actions with high executions times can cause dialog systems to react slowly due to high latency and high latex. |
| Approach: | They propose a model-agnostic method to achieve high quality in processing incrementally produced partial utterances. |
| Outcome: | The proposed method improves the metric F1-score by 47.91 percentage points . the proposed method can be used to create low-latency natural language understanding components on ATIS datasets. |
Multi-Task Networks with Universe, Group, and Task Feature Learning (P19-1)
Copied to clipboard
| Challenge: | In multi-task learning, multiple related tasks are learned together. |
| Approach: | They propose methods that take advantage of natural groupings of related tasks . they propose parallel and serial architectures that can learn different feature spaces . |
| Outcome: | The proposed methods improve performance on natural language understanding (NLU) tasks. |
An Evaluation of Neural Machine Translation Models on Historical Spelling Normalization (C18-1)
Copied to clipboard
| Challenge: | In this paper, we apply different NMT models to the problem of historical spelling normalization for five languages . we find that NMT model is much better than SMT in terms of character error rate . |
| Approach: | They propose to use NMT models to solve the problem of historical spelling normalization in five languages. |
| Outcome: | The proposed method improves historical spelling normalization for five languages. |
Event Detection with Neural Networks: A Rigorous Empirical Evaluation (D18-1)
Copied to clipboard
| Challenge: | Neural network models have been the most successful for event detection, but they ignore syntactic relationships in the text. |
| Approach: | They propose a GRU-based model that combines syntactic information along with temporal structure through an attention mechanism. |
| Outcome: | The proposed model is competitive with existing models on a ACE2005 dataset. |
Training-free Neural Architecture Search for RNNs and Transformers (2023.acl-long)
Copied to clipboard
| Challenge: | Neural architecture search (NAS) has allowed for the automatic creation of new and effective neural network architectures. |
| Approach: | They develop a new NAS metric that predicts the trained performance of an RNN architecture and significantly outperforms existing NAS metrics. |
| Outcome: | The proposed metric outperforms existing training-free metrics on the NAS-Bench-NLP benchmark. |
An In-depth Analysis of Implicit and Subtle Hate Speech Messages (2023.eacl-main)
Copied to clipboard
| Challenge: | Explicit hate speech is more easily identifiable by recognizing hateful words, but subtle messages are harmful . subtle messages contain linguistically subtle and implicit forms of HS, such as circumlocution, metaphors and sarcasm . social media have faced pressure from civil rights groups demanding to monitor and limit online hate speech . |
| Approach: | They propose to use a fine-grained definition of implicit and subtle messages to detect HS . they then experiment with neural network architectures to detect subtle content . |
| Outcome: | The proposed models perform satisfactory on explicit messages, but fail to detect subtle content. |
Data-to-text Generation with Entity Modeling (P19-1)
Copied to clipboard
| Challenge: | Recent approaches to data-to-text generation have shown great promise thanks to the use of large-scale datasets and the application of neural network architectures which are trained end-to end. |
| Approach: | They propose an entity-centric neural architecture for data-to-text generation which uses hierarchical attention to create entity-specific representations which are dynamically updated. |
| Outcome: | The proposed model outperforms baselines in automatic and human evaluation on the RotoWire benchmark and a five-times larger dataset on the baseball domain. |
A Diagnostic Study of Explainability Techniques for Text Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing explainability techniques that can be produced post-hoc with already trained models are lacking a definitive guide on how to choose one given a particular task and model architecture. |
| Approach: | They propose to use a list of diagnostic properties to evaluate existing explainability techniques to compare them with human annotations of salient input regions. |
| Outcome: | The proposed list compares a set of explainability techniques on downstream text classification tasks and neural network architectures. |
An Empirical Study of the Downstream Reliability of Pre-Trained Word Embeddings (2020.coling-main)
Copied to clipboard
| Challenge: | Pre-trained word embeddings have been shown to improve the performance of neural networks across a wide variety of tasks. |
| Approach: | They propose two new metrics to understand the downstream reliability of word embeddings. |
| Outcome: | The proposed model can improve performance with slight changes to the training data, but it can also fail with multiple neural network architectures. |
LABO: Towards Learning Optimal Label Regularization via Bi-level Optimization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for regularizing deep neural networks rely on weight decay, dropout, batch/layer normalization to converge faster and generalize. |
| Approach: | They propose a framework for training with label regularization which includes conventional LS but can also model instance-specific variants. |
| Outcome: | The proposed approach consistently yields better results than conventional regularization on seven machine translation and three image classification tasks while maintaining training efficiency. |
Training for Gibbs Sampling on Conditional Random Fields with Neural Scoring Factors (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in NLP focus on simple approaches to model the output label space . graphical models are often limited to (heuristic) greedy search and its variants . |
| Approach: | They propose an approach for efficiently training and decoding hybrids of graphical and graphical models based on Gibbs sampling. |
| Outcome: | The proposed approach improves on Dutch and Dutch with graphical models . the proposed model improves over a strong baseline on three languages . |
Relating Simple Sentence Representations in Deep Neural Networks and the Brain (P19-1)
Copied to clipboard
| Challenge: | Existing deep learning models for natural language processing are not fully studied. |
| Approach: | They investigate whether deep recurrent models learn sentences against those encoded by the brain and whether there is any correspondence between hidden layers of these models and brain regions when processing sentences. |
| Outcome: | The proposed models can be used to synthesize brain data and improve subsequent stimuli decoding accuracy. |
How poor is the stimulus? Evaluating hierarchical generalization in neural networks trained on child-directed speech (2023.acl-long)
Copied to clipboard
| Challenge: | LSTMs and Transformers perform well at capturing the surface statistics of child-directed speech, but both model types generalize in a way consistent with an incorrect linear rule than the correct hierarchical rule. |
| Approach: | They train LSTMs and Transformers on text from the CHILDES corpus and evaluate what they learn about English yes/no questions. |
| Outcome: | The proposed models perform well at capturing the surface statistics of child-directed speech, but generalize more consistent with an incorrect linear rule than the correct hierarchical rule. |
Neural Architectures for Nested NER through Linearization (P19-1)
Copied to clipboard
| Challenge: | a nested named entity recognition (NER) is a set of entities that can overlap and be labeled with more than one label. |
| Approach: | They propose two neural network architectures for nested named entity recognition . they propose to model nesting entities as multilabels and predict a sequence-to-sequence problem . |
| Outcome: | The proposed methods outperform the state-of-the-art on four corpora . the proposed models also improve on the recently published contextual embeddings . |
The Unstoppable Rise of Computational Linguistics in Deep Learning (2020.acl-main)
Copied to clipboard
| Challenge: | a quarter century ago, linguists assumed that language knowledge needed to be innate . but vector-space representations and machine learning algorithms are much more powerful than was thought . |
| Approach: | They trace the history of neural networks applied to natural language understanding tasks . they argue that Transformer is not a sequence model but an induced-structure model . |
| Outcome: | The proposed model is not a sequence model but an induced-structure model, the authors argue . they argue that the nature of language has had a profound impact on progress in machine learning . |
Argument-based Detection and Classification of Fallacies in Political Debates (2023.emnlp-main)
Copied to clipboard
| Challenge: | Fallacies are arguments that employ faulty reasoning, causing inaccurate conclusions and invalid inferences . ad hominem fallacy is one of the most common fallacy labels used in political debates despite its use in many scenarios . |
| Approach: | They extend the ElecDeb60To16 dataset of U.S. presidential debates annotated with fallacious arguments by incorporating the most recent Trump-Biden debate. |
| Outcome: | The proposed method extends the ElecDeb60To16 dataset of U.S. presidential debates annotated with fallacious arguments . |
Systematic Comparison of Neural Architectures and Training Approaches for Open Information Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Open information extraction (OIE) is a method for extracting facts from text in structured format . alternative formulations allow for longer tuples, but most work focuses on binary predicates only. |
| Approach: | They propose to extract facts from natural language text and represent them as structured triples . they compare different neural network architectures and training approaches . |
| Outcome: | The proposed approach improves the currently best models on the OIE16 benchmark by 0.421 F1 score and 0.420 AUC-PR . |
Improving Neural Metaphor Detection with Visual Datasets (2020.lrec-1)
Copied to clipboard
| Challenge: | a new method for metaphor detection uses text from visual datasets to identify words . a metaphor is a complex interaction between two terms, creating an "implicationcomplex" |
| Approach: | They propose a technique for sampling text from visual datasets to create a visibility word embedding. |
| Outcome: | The proposed method improves on previous approaches that use more complex neural networks and richer linguistic features for verb classification. |